<!doctype html public "html2.0">
<html>
<head>
<title>IBM TSpaces User's Guide</title>
<!-- created by: Toby Lehman, 25-Jan-1998 -->
<meta name="abstract" content="TSpaces User Manual">
<meta name="keywords" content="TSpaces Java TupleSpace Database">
<meta name="owner"    content="toby@almaden.ibm.com">
<meta name="review"   content="19980125">
<meta name="security" content="public">
</head>

<! Set white background with red and purple links ->
<body bgcolor="#ffffff" text="#000000" link="#CC0000" alink="#ff3300" vlink="#330099">
<! Include the Masthead -> 
<center>
<img src="gifs/masthead.gif" alt="TSpaces" ALIGN = "middle" BORDER = 0 width=600 height=54>
    
</center>


<center>
<H1>IBM TSpaces User's Guide </H1>
</center>
<br clear=right>
<!----------------------------------------------------------------->
<center> <h2><a name="over">  Overview </h2> </center>
<!----------------------------------------------------------------->
<p>
TSpaces is a new software package that provides a common set of
services for a network of hetergeneous computers and operating systems.
It offers group communication services, database services,
URL-based file transfer services and
event notification services.  Compared to the systems whose function
it emulates, TSpaces is tiny.  And so, given its small
footprint, it is an ideal solution for bringing network services
to small and embedded systems.
<center>
<img src="gifs/TSGroup.gif"
    alt="A TS Network with various clients"
    ALIGN = "center" BORDER = 0 >
</center>
<p>
From a TSpaces client point of view, being connected to TSpaces is like
being connected to the perfect assistant.  It will remember things
for you, it will carry out any tasks that you assign it, it will report
incoming messages and deliver outgoing messages, and it will notify
you of any events in which you're interested.  By adding additional
client applications, it is possible to use TSpaces as a unversal print
service, email service, pager service, remote control service, and so on.
<p>
Since it is written in Java, T Space client applications can be loaded
dynamically into any network attached computer and run.  In addition,
the T Space server, also written in Java, can be updated with new function
and capabilities while it is running, thus avoiding costly downtime
for system upgrades.

<!----------------------------------------------------------------->
<h2><a name="background"> 1.0 Background</h2>
<!----------------------------------------------------------------->
<p>
Network applications have been around for 15 years and yet, in that
time, convenient middleware for sharing data across applications has
not been plentiful.  Sure, there are numerous tools and utilities for
getting programs and devices to communicate, but so far there has been
no network-oriented, easy-to-use data repository that is widely available.
The packages have been too low level (e.g., TCP/IP sockets) to cover much
of the needed functionality or too heavy weight (e.g., "real" database
systems) to be really convenient.
<p>
Of course, these trends are not without reason.  It is hard to build
really useful middleware across heterogeneous platforms.  One of the hardest
problems is just dealing with the
representations of objects (e.g., C and C++ objects), which often have
differing structure layouts per compiler and per platform.  Some
attempts have been made to solve this problem alone (e.g., CORBA),  but
so far there's been no clear success story at the communication 
layer, much less at the middleware layer.
<p>
Now, the story is even worse.
Besides the mediocre mechanisms involved, the mode in which distributed
network applications run is changing.  In the early years, it was a complete
success simply to have two programs start execution, exchange a packet of data,
then terminate.  The data didn't live any longer than the programs.  Now,
this is changing.  As the Database Developers can attest, data
is living longer than ever.  Even data that is not in a heavy weight
repository often outlives the program instance that generated it.  As
a result, today's network-oriented application programs require data
repositories that can hold the data beyond the life of the generating
applications.  In addition, these repositories must allow application
programs to attach to the data, execute some logic on it, and terminate,
all in an ad hoc basis.
<p>
What do we have that will meet these needs?  The solution has several
pieces, but luckily, they are all falling into place.   One of the larger
pieces is Java.  Java has single handedly solved the communication part
of the problem.  Another major piece is TSpaces.  TSpaces, coupled with
Java's dynamic and ubiquitous nature, creates a uniform platform on which
we can create applications to perform practically any service desired.

"Where did TSpaces come from?" you ask.  Well, it's an interesting
story.  Mostly, it started out as the marriage of a tuplespace system
with a database system.


<!----------------------------------------------------------------->
<h3><a name="somethingold"> 1.1 Something Old</h3>
<!----------------------------------------------------------------->
<p>
In the mid-1980's, David Gelernter, a professor at Yale University,
created a project called LINDA
[Carriero 84], [Gelernter 82], [Gelernter 84], [Gelernter 85].
LINDA was a programming language
designed to address the communication problems of parallel programming.
Part of this project was a concept known as <b>Tuplespace</b>.
Tuplespace embodied three main principles:
<ul>
<li> Anonymous communication
<li> Universal associative addressing
<li> Persistent data
</ul>
Although the LINDA Tuplespace was originally intended as a global
communication buffer for parallel programs (which are concerned mostly with
synchronization and high performance), we found that it also works well for
distributed programs which often need a ubiquitous persistent
communication buffer.
<p>
The central concept of a LINDA Tuplespace was surprisingly simple -- agents
post unstructured
tuples to a univerally visible Tuplespace using an "Out" command,
consume tuples using an "In" command and read tuples using a "Read" command.
LINDA was a bit hit in the parallel programming community, but it has
not enjoyed much visibility outside of that particular research area.
<p>

<!----------------------------------------------------------------->
<h3><a name="somethingnew"> 1.2 Something New</h3>
<!----------------------------------------------------------------->
<p>
In 1996 (which used to be "new") we had heard that SUN and others were
showing interest in LINDA.
SUN has more recently publicized an internal Tuplespace project,
written in Java, called <b>JavaSpaces</b>.
We experiemented with the Tuplespace idea -- implementing a simple
prototype in Java.  And, almost immediately, we saw 
a number unexpected opportunities. We saw a ubiquitous platform with
powerful object oriented types to express any type of data, dynamic class
loading so that new types could be loaded on the fly, and most importantly,
new commands (operators) could be defined to the server.  Thus, 
a Tuplespace Server can be "taught" how to perform more
than the initial OUT, IN, READ commands.
<p>

<!----------------------------------------------------------------->
<h3><a name="somethingborrowed"> 1.3 Something Borrowed</h3>
<!----------------------------------------------------------------->
<p>
Given that we're in the database research group at IBM, we naturally
thought of TSpaces as being a lightweight data <b>repository</b>
in addition to being a global communication buffer, so we basically
"borrowed" some of the database technology that we had sitting
around.  Interestingly,
SUN claims that Javaspaces is most definitely <b>NOT</b>
a persistent data repository.  Anyway, being a repository, TSpaces needed
several database features for data integrity and search speed,
such as transaction support, index support and a simple query language
(much simpler than SQL, but better than the overly restrictive "formal"
tuple queries offered by LINDA).  In addition, the
TSpaces server can also deal with an arbitrary collection of Java types,
as clients wishing to add new types just define them to the server and
then use them.
<p>

<!----------------------------------------------------------------->
<h3><a name="somethingblue"> 1.4 Something Blue</h3>
<!----------------------------------------------------------------->
<p>
Our favorite choice for the name of this project was "Bluespaces"
(kind of a tongue in cheek play on the name "Javaspaces"), but
it seems that, as a corporation, we're trying not to overuse the
"blue" name.   So, as a second choice, we ended up with "TSpaces".

<!----------------------------------------------------------------->
<h2><a name="using"> 2.0 Using TSpaces</h2>
<!----------------------------------------------------------------->
<p>
TSpaces is network middleware.  Middleware has no face, no
frontend to speak of.  It is the applications, the middleware
<i>clients</i>, that
users see.  Thus, the usefulness of the middleware is determined by
the usefulness of the client applications.   Fortunately, the function
offered by TSpaces is sufficiently powerful that it is easy to write
meaningful and useful applications.   Look at the
<a href="examples.html"> TSpaces example programs</A>.  We wrote
these just to demonstrate how TSpaces programs can be written, but
it turns out that we use some of these programs for everyday work.
<p>
TSpaces has many faces and these different faces serve different
application needs.  You can think of TSpaces as any or all of
the following:
<ul>
<li> A global persistent communication buffer
<li> A lightweight database
<li> A queue manager (a simple form of IBM's MQ Series)
<li> A dynamic computation engine
<p>
One of the benefits of being attached to a TSpaces server is that
you get the services of a real database system (queries, transactions,
persistent data, etc) without the drudgery
of dealing with Relational Database System design or SQL queries.
To insert data into a T Space (i.e. a database), you just drop it
in with a write statement -- no schema, no table definitions, no
ugly JDBC/ODBC statements.
<p>
<!----------------------------------------------------------------->
<h3><a name="model"> 2.1 The TSpaces Model </h3>
<!----------------------------------------------------------------->
<p>
The model is simple.
There are clients and there are servers.
<center>
<img src="gifs/TSClientServer.gif"
    alt="A TS Server and two clients"
    ALIGN = "center" BORDER = 0  >
</center>

Clients read and write data from/to a server
with simple method calls, like this:
<pre>
// Create the initial tuplespace
Tuplespace ts = new Tuplespace(  spaceName, serverName);
// Write some data, tagged "mydata1"
ts.write("mydata1", dataInstance);
</pre>
That's it.  Easy. No table definitions, no clumsy SQL insert statements.
No decomposing your complex datatype into SQL primitive ints, floats
and character types.  Then, if you want to read this data from the
same or a different application, you just read it.
<pre>
// Read the specific data record
resultTuple = ts.read("mydata1", DataInstance);
// Read ALL the records of that type
resultTupleSet = ts.scan(String, DataInstance);
</pre>
NOTE: although a client can talk to several servers simultaneously,
in the current release, servers do not share information (i.e.
we do not currently support caching or replication across servers and
servers do not cooperate on any single transaction).

<!----------------------------------------------------------------->
<h3><a name="server"> 2.2 The TSpaces Server</h3>
<!----------------------------------------------------------------->
<p>
Please consult the
<a href="ProgrGuide.html"> TSpaces Programming Guide</a>
for the instructions on invoking the TSpaces server.
<p>
TSpaces servers can be run everywhere.   We find it convenient to
run them locally (to coordinate a few office machines) as well as
run them in department servers (for wider range services).  For
example, we have designated a default "TSpaces Server" machine that
is chosen by the client in the event that the client does not
specify a server.   So, just like "NameServer" and "PrintServer"
are well-known aliases, "TSServer" has been added to the list
in our environment.
<p>
However, there is no pain involved in running your own server, since
they require basically zero administration.
For example, one of our users, Joe, uses six different machines
(yes, we're very proud of Joe).  Joe
designates one machine to be the local server, and the other 
machines just refer to that machine for the TSpaces service.
We expect that as applications appear that stay resident on users'
desktops, TSpaces servers will become common place as background
processes.
<p>

<!----------------------------------------------------------------->
<h3><a name="client"> 2.3 A TSpaces Client</h3>
<!----------------------------------------------------------------->
<p>
A TSpaces client program is just any old program that makes calls
to the TSpaces server.  Once the server is running, the client
application program just runs.  So, for example, if you're running
the BlueClipboard application, you just fire it up and it runs.
Please consult the
<a href="ProgrGuide.html"> TSpaces Programming Guide</a>
for the instructions on invoking the various TSpaces applications.

<p>
<!----------------------------------------------------------------->
<h2><a name="basicarch"> 3.0   Basic Architecture </h3>
<!----------------------------------------------------------------->
<p>
In this section we briefly discuss the basics of TSpaces.
<p>
<!----------------------------------------------------------------->
<h3><a name="term"> 3.1   Terminology </h3>
<!----------------------------------------------------------------->
<p>
The client interface is very simple.  A client creates an instance
of a tuplespace, then uses the methods of that instance to read
and write tuples.  Tuples are just Java vectors of fields, which are
basically type/value pairs.

<p>
<!------------------------------------>
<h4>3.1.1 Field class</h4>
<!------------------------------------>
<p>
The <b>Field</b> class is the most basic component of the Tuplespace data
structure hierarchy.  A field contains:
<ul>
 <li> Type
 <li> Value
 <li> Field Name (optional): Specifying a field name implies that this
     field will be indexed (and searched).
</ul>

<p>
<!------------------------------------>
<h4>3.1.2 Tuple class</h4> 
<!------------------------------------>
<p>
A Tuple is a vector of Fields.
<p>
The <b>SuperTuple</b>
class is where the interesting functionality of tuples is implemented,
but it's an abstract class, so clients should use either the <b>Tuple</b>
or the <b>SubclassableTuple</b>.

<p>
<!----------------------------------------------------------------->
<h3><a name="client"> 3.2   Client Side Basics </h3>
<!----------------------------------------------------------------->
<p>
Communication --- the communication protocol used between the client and
server and the communication interface on both the client and the server
<p>
The client and server communicate with Sockets and ObjectStreams. 

<p>
<!------------------------------------>
<h4>3.2.1 TupleSpace class</h4>
<!------------------------------------>
<p>
The TupleSpace Class is the main structure for attaching to the
tuplespace community. It is the TupleSpace methods that an application
uses to send and receive tuples from the shared network repository.
<p>
A TSpaces server contains many Tuplespaces -- potentially billions.
It is up to the application writer to decide whether to use one or many
tuplespaces for a particular application.  However, it is important to
know that it's really hard to run out of spaces, so there's no reason to
try to fit too many different tuple types into a single space.

<!------------------------------------>
<h4>3.2.2 TupleSpace constructors</h4>
<!------------------------------------>

There are a lot of constructors.  The ones that aren't documented just
add username and password to the preceding one.
Interesting points: if you don't specify an
accessPermissions tuple or a configuration tuple, the server will not
be contacted.

<!------------------------------------>
<h4>3.2.3 TupleSpace methods</h4>
<!------------------------------------>
<p>

Interesting ones: write, read, take, waitToRead, waitToTake, scan,
consumingScan, count, countN, delete, deleteAll, and Rhonda.

<p>
<!----------------------------------------------------------------->
<h3><a name="server"> 3.3   Server Side Basics </h3>
<!----------------------------------------------------------------->

<!------------------------------------>
<h4>3.3.1 Command Control Flow</h4>
<!------------------------------------>
<p>
The TSpaces server is composed of two main layers.  The bottom layer
comprises the basic tuple management.  This is where tuple sets are stored,
updated, indexed and scanned.  The interface to this layer is the
<b>Tuple Management API</b>.  The top layer comprises the operator component,
which is responsible for operator registration and handler, the implementation
of an operator, management.
<p>
<img src="gifs/BSPicture.gif"
     alt="Server Structure"
     ALIGN = "left" BORDER = 0 height=480>
<p>
To execute a regular operator, such as read, the server performs the following
steps:
<ul>
<li> The Top Level Command Handler reads the incoming message and dispatches
    the appropriate operator lookup mechanism.
<li>  The operator lookup checks the operator registry to see if the
    operator is valid (it is defined with an associated handler), then,
    assuming it is valid, locates the proper handler.
<li> The Command Handler launches the operator handler, with the original
    client message as the input.
<li> The operator handler executes, then returns an answer (if one is
    expected) to the Command Handler.
<li> The Command Handler returns to the client.
</ul>
<p>
In more specific terms, here's what happens:
<ul>

<li> The main server class, <b>TSServer</b>
accepts connection from client, creates a TSIOHandler and passes off
the socket (and goes back to listening for connections).

<li> The main connection class, <b>TSIOHandler</b>
creates a TSDispatch object for the command, and waits for a 
response from the TSDispatch.

<li> <b>TSDispatch</b> locates the TupleSpace referenced by the
client message.  The TupleSpace (specifically "TS" in Java Class
language) has associated with it a database manager (TSDB).  The
specific command sent by the client has associated with it an
operator family, which translates to an operator handler <b>factory</b>,
(TSFactory).

<li> TSFactory locates the the specific instance of TSHandler (which
is an interface) for the operator, hands the handler to TSDispatch,
who invokes the handler to execute the operator.  The results of the
handler are passed back to TSIOHandler, who then sends the answer
back to the client.

</ul>

<p>
<!------------------------------------>
<h4>3.3.2 Blocking Commands</h4>
<!------------------------------------>
<p>
When an application issues a <b>WaitToRead</b> or <b>WaitToTake</b> call,
and the data is not yet there on the server, the application blocks on the
call until an answer is returned.  When a tuple arrives on the server
that matches the <b>Read</b> or <b>Take</b> query, it is sent to the
client and the application resumes.
<p>
Although the easy way to implement this would be to allocate a thread
on the server for each user connection (which is what we did in our
initial prototype implementation), this results in far too many threads
when there are several outstanding application requrests.  Therefore,
under the covers, TSpaces uses a callback mechanism to contact the
client when a tuple matching the outstanding query is present.
That way, we do not need to keep the server thread suspended.
And, unlike SUN's Javaspaces who also use notification, we lock the tuple
before issuing the callback so that the user is guaranteed of getting
the tuple when receiving the callback.


<p>
<!------------------------------------>
<h4>3.3.3 Persistence</h4>
<!------------------------------------>
<p>
There are two planned implementations of the tuple database.  The
memory-resident database (Light TSpaces) uses simple file-level
persistence for tuples and indexes.  The heavy-weight solution is
to use IBM's DB2 (Deep TSpaces) as the main store.  With DB2,
we get an industrial grade transaction-based repository with
inifinite storage capability.
Currently, only the file-level persistence is supported.

<p>
<!------------------------------------>
<h4>3.3.4 Access Control </h4>
<!------------------------------------>
<p>
Access control is on a TupleSpace level; each operation defined on a
TupleSpace has an associated list of <i>AccessAttributes</i> that must
be satisfied by any client trying to execute that operation. For
example, the <b>take</b> operation has <i>Read</i> and <i>Write</i> 
<b>AccessAttributes</b>, and the <b>addHandler</b> operation has the
<i>Admin</i> <i> AccessAttribute</i>.

<!----------------------------------------------------------------->
<h3><a name="database"> 3.4   The Server Database </h3>
<!----------------------------------------------------------------->
<p>
Light TSpaces employes a memory-resident database system for
managing the tuples.  The internal structures of this memory-resident
database system are direct descendents
from the Starburst Main Memory Manager [Starburst 92].  And, along
with the tuple management, there are the Starburst indexes as well.
The T Tree index [Lehman 86a, Lehman 86b] and the Modified Linear
Hash index structures are used as well for general index search.
The T Tree has gone through some modifications to allow both regular
(single and multi-attribute, unique and non-unique) indexing and
inverted text indexing.


<hr>
<!----------------------------------------------------------------->
<h2><a name="bib"> 4.0   References </h2>
<!----------------------------------------------------------------->
<p>
<ul>
<li> [Carriero 84] Nicholas Carriero, David Gelernter:
    Linda in Context. CACM 32(4): 444-458 (1984) 
<li> [Gelernter 82] David Gelernter, Arthur J. Bernstein:
    Distributed Communication via Global Buffer. PODC 1982: 10-18 
<li> [Gelernter 84] David Gelernter:
    A Note on Systems Programming in Concurrent Prolog. SLP 1984: 76-82 
<li> [Gelernter 85] David Gelernter:
    Generative Communication in Linda. TOPLAS 7(1): 80-112 (1985) 
<li> [Lehman 86a] Tobin J. Lehman, Michael J. Carey:
    Query Processing in Main Memory Database Management Systems.
    SIGMOD Conference 1986: 239-250 
<li> [Lehman 86b]  Tobin J. Lehman, Michael J. Carey:
    A Study of Index Structures for Main Memory Database Management Systems.
    VLDB 1986: 294-303 
<li> [Starburst 92]  Tobin J. Lehman, Eugene J. Shekita, Luis-Felipe Cabrera:
    An Evaluation of Starburst's Memory Resident Storage
    Component. TKDE 4(6): 555-566 (1992) 
</ul>



<! ------------------------------------------------------------------------->
<hr>
<! ------------------------------------------------------------------------->
[
<a href="../index.html" target=_top>
TSpaces home page </a>
| 
<a href="ProgrGuide.html" >
Programmer's Guide </a>
]
</body>
